In this paper, a new method for resolution enhancement of single document images is presented. The proposed method is example based using an example set of low-resolution and high-resolution training patches. According to the Bayes rule, one function is considered as the likelihood or data-fidelity term that measures the fidelity of the output high-resolution to the input low-resolution image. As well, three other functions are considered as the regularization terms containing the prior knowledge about the desired high-resolution document image. Three priors which are fulfilled by the regularization terms are bimodality of document images, smoothness of background and text regions, and similarity to the patches in the example set. By minimizing these four energy functions through the iterative procedure of asynchronous sequential Gradient descent, the HR image is reconstructed. Instead of synchronous minimization of the linear combination of these functions, they are minimized in order and according to the gradual changes in their values and in the updating HR image. Therefore, determining the coefficients of the linear combination, which are variable for input images, is no longer required. In the experimental results on twenty document images with different fonts, at different resolutions, and with different amounts of noise and blurriness, the proposed method achieves significant improvements in visual image quality and in reducing the computational complexity.

Imposing a constraint on the Gradient descend algorithm for the purpose of neural network training with limited weights has several applications such as network transparency, reducing network volume in terms of storage, increasing the level of generalizability. In addition, it speeds up convergence and finds a more accurate answer. In this paper, using the kernel trick as a method for imposing various constraints on the training algorithm, 21 different constraints have been compared those 16 constraints of which are novel and inspired by the existing uncertainty in biological neural networks. There have been no data augmentation or regularization techniques are used to clearly show the effect of each constraint functions. To solve the classification problem of MNIST, CIFAR-10 and CIFAR-100 datasets for each constraint function, 63 experiments have been done. The results show constraint functions have different impact on solving each dataset and specially our biologically inspired constraints may lead to train a more accurate neural network than the other constraints.

The present study compares and predicts the predictive ability of the capital market based on the learning pattern of the Levenberg-Marquardt algorithm, the Gradient descent and the ARIMA algorithm. For this purpose, market data were used in the period from 1394 to 1397, and more than 75% of these data were used as training data prior to 1397, and one year end data were used as data. The results of the evaluation of the research data show that artificial neural networks have a high capacity for price prediction. The results also showed that in both training data series from 1394 to 1396 and experimental of 1397 the comparison of the results and performance of ARIMA neural networks (ARIMA) showed that the neural network had higher predictive power in Comparing with the performance and prediction accuracy of two types of neural networks with the Levenberg-Marquardt learning algorithm and the Gradient descent learning algorithm using the Levenberg-Marquardt learning algorithm has been able to increase the neural network prediction accuracy And reduce its error, so, the results of the present study show, the Levenberg-Marquardt learning algorithm improves the predictive power of the neural network.

The purpose of this study is to analyze the performance of Back propagation algorithm with changing training patterns and the second momentum term in feed forward neural networks. This analysis is conducted on 250 different words of three small letters from the English alphabet. These words are presented to two vertical segmentation programs which are designed in MATLAB and based on portions (1/2 and 2/3) of average height of words, for segmentation into characters. These characters are clubbed together after binarization to form training patterns for neural network. Network was trained by adjusting the connection strengths on each iteration by introducing the second momentum term. This term alters the process of connection strength fast and efficiently. The conjugate Gradient descent of each presented training pattern was found to identify the error minima for each training pattern. The network was trained to learn its behavior by presenting each one of the 5 samples (final input samples having 26 × 5 = 130 letters) 100 times to it, thus achieved 500 trials indicate the significant difference between the two momentum variables in the data sets presented to the neural network. The results indicate that the segmentation based on 2/3 portion of height yields better segmentation and the performance of the neural network was more convergent and accurate for the learning with newly introduced momentum term.

Doze and sleep mechanisms are the most common energy-saving solution in GPON networks. Sleep duration is the critical value in the energy-saving domain, and it will affect the QoS metrics with inappropriate value. In this paper, a new energy-saving mechanism is proposed using an optimal point based on Gradient descent that calculates sleep duration and keeps QoS metrics acceptable. The historical value of average delay and packet, drop ratio, ONU buffer, and bandwidth request parameters are used as input, and the sleep duration value is calculated. The simulation results show that the proposed method saves up to 17% energy in GPON and keeps the network’s QoS in an acceptable domain.

The stability of the training process in the identification of nonlinear systems is one of the foremost issues in control research. This paper studies the training stability of an interval type 2 adaptive neuro-fuzzy Inference system (IT2ANFIS) as an identifier through a newfound Lyapunov function. Lyapunov stability analysis is conducted on the training of IT2ANFIS, when the premise and the consequent of the system are trained with the Gradient descent algorithm and the Kalman Filter, respectively. Therefore, using the proposed stability analysis, the permissible limits for the adjustable parameters of the algorithms are applied to the algorithms to maintain the stability of the identification process. According to the stability analysis of this study, wide ranges of adaptive limits are obtained for the adjustable parameters of the algorithms. Besides, the simulation results show that when the permissible limits are chosen based on the proposed stability analysis, the identification process is stable with acceptable performance. The proposed method outperforms other methods in terms of root mean square error, simulation time, and its less stagnation in the trap of local minimums when it is utilized in the training of the Mackey-Glass chaotic time series and a nonlinear plant with stochastic data sets.

A large amount of research in the field of online learning has focused on the problem of overcoming catastrophic forgetting, and few research studies have focused on classifying the data stream with appropriate accuracy and running time. On the other hand, due to the volume and type of data stream, many traditional machine learning algorithms do not have the necessary efficiency when faced with it. Thus, in this paper, a novel model using reinforcement learning and the stochastic Gradient descent algorithm is presented for the classification stream data with appropriate accuracy and running time. One of the important features of reinforcement learning is that the agent can adapt its behaviour gradually to the changes that occur and gradually add to its previous knowledge. In this research, because of the use of reinforcement learning and the definition of reward, the agent has a better performance in the environment. The proposed algorithm has been tested on various data, including the dataset of human activity recognition, and compared with several incremental algorithms in terms of accuracy and running time. According to the experimental results, the proposed algorithm has the best performance in terms of both accuracy and running time compared to other incremental algorithms.

In this paper, an efficient GV1-CG is developed to optimizing the modified conjugate Gradient algorithm by using a new conjugate property. This is to to increase the speed of the convergence and retain the characteristic mass convergence using the conjugate property. This used property is proposed to public functions as it is not necessary to be a quadratic and convex function. The descent sharp property and comprehensive convergence for the proposed improved algorithm have been proved. Numerical results on some test function indicate that the new CG-method outperforms many of the similar methods in this field.

In this paper, we present a new hybrid conjugate Gradient method for unconstrained optimization that possesses sufficient descent property independent of any line search. In our method, a convex combination of the Hestenes-Stiefel (HS) and the Fletcher-Reeves (FR) methods, is used as the conjugate parameter and the hybridization parameter is determined by minimizing the distance between the hybrid conjugate Gradient direction and direction of the three-term HS method proposed by M. Li (\emph{A family of three-term nonlinear conjugate Gradient methods close to the memoryless BFGS method,} Optim. Lett. \textbf{12} (8) (2018) 1911--1927). Under some standard assumptions, the global convergence property on general functions is established. Numerical results on some test problems in the CUTEst library illustrate the efficiency and robustness of our proposed method in practice.